Skip to content

PYTHON-5413 Handle flaky tests #2395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Jul 1, 2025
Merged

Conversation

blink1073
Copy link
Member

@blink1073 blink1073 commented Jun 20, 2025

The goal was to get the following to pass on multiple restarts:

  • MongoDB 8.0 with pypy
  • All of the Windows and MacOS standard tests
  • Encryption crypt_shared with pypy
  • Free-threading builds

Since these are the most flaky builds in the waterfall currently. The current build has passed three times for the above variants.

@blink1073 blink1073 marked this pull request as ready for review July 1, 2025 18:01
@blink1073 blink1073 requested a review from a team as a code owner July 1, 2025 18:01
@blink1073 blink1073 requested a review from ShaneHarvey July 1, 2025 18:01
Copy link
Member

@ShaneHarvey ShaneHarvey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we only enable flakey in CI and not locally? I worry having it run by default locally will make it more difficult to debug test failures.

async def test_timeout_in_multi_batch_bulk_write(self):
if sys.platform != "linux":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we do this only in Evergreen CI?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated this and added logic/docs around enabling and disabling the flaky decorator.

@blink1073 blink1073 requested a review from ShaneHarvey July 1, 2025 19:11
@blink1073 blink1073 merged commit 0b2900d into mongodb:master Jul 1, 2025
78 of 79 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants